Papers with natural language inference task
AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models (2022.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches to NLP tasks rely on pre-trained language models, but some do not. |
| Approach: | They propose a natural language inference task to test pre-trained language models for logical reasoning capabilities. |
| Outcome: | The proposed language model performs better than other models across logical connectives and reasoning domains, but is sensitive to lexical and syntactic variations in the realisation of logical statements. |
Comparison and Combination of Sentence Embeddings Derived from Different Supervision Signals (2022.starsem-1)
Copied to clipboard
| Challenge: | Existing methods to derive sentence embeddings have not been well understood what properties are captured in the resulting sentences depending on the supervision signals. |
| Approach: | They propose to combine two types of sentence embedding methods with similar architectures and tasks to investigate their properties. |
| Outcome: | The proposed methods perform better on unsupervised and downstream tasks than the proposed methods on untrained STS tasks and probing tasks. |
Enhancing Clinical BERT Embedding using a Biomedical Knowledge Base (2020.coling-main)
Copied to clipboard
| Challenge: | Domain knowledge is important for building Natural Language Processing (NLP) systems for low-resource settings, such as in the clinical domain. |
| Approach: | They propose a joint method for adding knowledge base information from the Unified Medical Language System (UMLS) into language model pre-training for some clinical domain corpus. |
| Outcome: | The proposed method outperforms existing models on three clinical domain tasks with no knowledge base information. |
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT (D19-1)
Copied to clipboard
| Challenge: | Pretrained contextual representation models have pushed forward the state-of-the-art on many NLP tasks. |
| Approach: | They propose to use a model that is pretrained on 104 languages for cross-lingual transfer. |
| Outcome: | The proposed model performs well on 5 NLP tasks covering 39 languages from various language families. |
Language Models for Lexical Inference in Context (2021.eacl-main)
Copied to clipboard
| Challenge: | Lexical inference in context (LIiC) is a variant of the natural language inference task focused on lexical semantics. |
| Approach: | They propose three approaches based on pretrained language models for LIiC . they propose a few-shot NLI classifier and a relation induction approach based upon handcrafted patterns expressing the semantics of lexical inference. |
| Outcome: | The proposed approaches outperform the previous state of the art and show their potential for LIiC. |
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing systems that make predictions and ask questions are unable to have a mutual exchange of opinions. |
| Approach: | They propose to use a dataset and computational framework to allow systems to have beneficial discussions with humans, improving the accuracy by 25 points on a natural language inference task. |
| Outcome: | The proposed system improves accuracy by 25 points on a natural language inference task. |
UDAPTER - Efficient Domain Adaptation Using Adapters (2023.eacl-main)
Copied to clipboard
| Challenge: | Using adapters, unsupervised domain adaptation (UDA) is more parameter efficient and requires large-scale data to be effective. |
| Approach: | They propose to add small bottleneck layers to each layer of a pre-trained language model to make it more parameter efficient by adding adapters. |
| Outcome: | The proposed methods outperform unsupervised domain adaptation methods such as DANN and DSN in natural language inference and sentiment classification tasks. |
Lessons from Natural Language Inference in the Clinical Domain (D18-1)
Copied to clipboard
| Challenge: | State of the art models with deep neural networks lack generalization capabilities in specialized domains where training data is limited. |
| Approach: | They propose a dataset annotated by doctors performing a natural language inference task grounded in the medical history of patients. |
| Outcome: | The proposed model outperforms existing models in the clinical domain by incorporating domain knowledge from external data and lexical sources. |
From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text (2021.acl-long)
Copied to clipboard
| Challenge: | a computational model for code-switching text is lacking in the corpus of real text. |
| Approach: | They propose a neural machine translation model to generate Hindi-English code-switched sentences using monolingual Hindi sentences. |
| Outcome: | The proposed model reduces perplexity on a language modeling task and improves on linguistic inference tasks. |
Transformation of Dense and Sparse Text Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to NLP to leverage sparsity have been limited due to the gap with dense representations. |
| Approach: | They propose a Semantic Transformation method to bridge dense and sparse spaces and propose supervised NLP tasks to use both spaces. |
| Outcome: | Experiments with classification tasks and natural language inference tasks show that the proposed method is effective. |
SDOH-NLI: a Dataset for Inferring Social Determinants of Health from Clinical Notes (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Social and behavioral determinants of health (SDOH) play a significant role in shaping health outcomes, and extracting these determinant from clinical notes is a first step to help healthcare providers systematically identify opportunities to provide appropriate care and address disparities. |
| Approach: | They propose a dataset that extracts social and behavioral determinants from clinical notes and uses them to form a natural language inference task. |
| Outcome: | The proposed dataset is based on publicly available notes and is more challenging than standard NLI benchmarks. |
Identifying and Explaining Discriminative Attributes (D19-1)
Copied to clipboard
| Challenge: | Existing word vector representation models lack latent features (dense vectors) identifying discriminative attributes can motivate the development of word vector models with finer semantics. |
| Approach: | They propose to use a word vector representation model to identify discriminative attributes by combining knowledge graphs with images to construct explicit vector spaces. |
| Outcome: | The proposed model performs comparable to state-of-the-art systems while providing full model transparency and explainability. |
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on optimal decision-making are limited and only consider individuals in isolation. |
| Approach: | They propose a task and corpus for learning alignments between machine and human preferences based on a gamified voting game . |
| Outcome: | The proposed task and corpus show that current state-of-the-art NLP models still leave much room for improvement. |
Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for textual emotion classification depend on domain and application scenario and need to be predefined . a natural language inference model with a flexible set of labels is difficult to develop . |
| Approach: | They propose to use the paradigm of zero-shot learning as a natural language inference task to generate a model with a flexible set of labels. |
| Outcome: | The proposed model is more robust across corpora than individual prompts and shows similar performance to the best prompt for a particular corpus. |
Learning with Different Amounts of Annotation: From Zero to Many Labels (2021.emnlp-main)
Copied to clipboard
| Challenge: | a lack of annotator agreement can hinder training of NLP systems . we propose a learning algorithm that can learn from training examples with zero, one, or multiple labels. |
| Approach: | They propose an annotation distribution scheme that assigns multiple labels to training examples . they propose a learning algorithm that can learn from training examples with different amount of annotation . |
| Outcome: | The proposed method achieves consistent gains in two tasks, suggesting distributing labels unevenly among training examples can be beneficial for many NLP tasks. |
Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly deployed in multilingual, real-world applications where user inputs introduce typographical errors. |
| Approach: | They propose a multilingual typo generation algorithm that simulates human-like errors based on language-specific keyboard layouts and typing behavior. |
| Outcome: | The proposed model can generate the correct answer ("500") under typos in English, German, and Russian. |
Verbing Weirds Language (Models): Evaluation of English Zero-Derivation in Five LLMs (2024.lrec-main)
Copied to clipboard
| Challenge: | Lexical-syntactic flexibility is a hallmark of English morphology . conversion involves placing a word with one part of speech in a non-prototypical context . |
| Approach: | They propose to test lexical-syntactic flexibility in the form of conversion . conversion is a process where a word with one part of speech is placed in a non-prototypical context . |
| Outcome: | The proposed task tests the ability of five language models to generalize over words with a non-prototypical part of speech. |